Identifying heterogeneous treatment effects for online single-session interventions for adolescent depression: a secondary analysis
Note: Press the button on the upper right of this file to show all codes.
Report: Download 2024_Comps_Part_III_Report.pdf. Git repo can be found here
1 Data preparation
First, load dataset and clean data as necessary.
1.1 Collapse categories
Re-code sexual orientation, race, family challenge variables by collapsing some categories with limited number of samples, and generating composite categories.
1.1.0.1 Recode sexual orientation
1.1.0.2 Recode family challenges
1.1.0.3 Recode race
1.2 Latent Class Analysis(LCA)
Apply LCA to find the latent class variables for gender identity and coping strategies, respectively.
1.2.1 Gender identity
1.2.1.1 Step 1: LCA model selection
Compare LCA models for different number of classes.
the LCA model with 3 latent classes has comparable \(G^2\), AIC, BIC. (could also use scree plot). therefore, a model with 3 latent class is selected for gender identity.
| Model | G2 | AIC | BIC |
|---|---|---|---|
| 2 | 1348.0556 | 7529.195 | 7683.297 |
| 3 | 924.7536 | 7135.893 | 7369.704 |
| 4 | 821.6347 | 7062.774 | 7376.293 |
| 5 | 616.5568 | 6887.696 | 7280.923 |
1.2.1.2 Step 2: classification error: averaged posterior probability(APP)
print APP, results suggest low classification error(APP>0.7).
## [1] 0.9648380 0.9863283 0.9650341
1.2.1.3 Step 3: Final latent class probabilities
The probability of belonging to a specific class is calculated for each individual, and each individual is classified to a specific class based on the max. posterior probability. Below is the results of the posterior probabilities for all subjects (prob. of being in class \(j\)).
1.2.1.4 Step 4: Assign label to each latent class (gener identity)
Visualize the probabilities of answering yes of each item by latent class (Pr(individual answers yes to an item \(|\) in class \(j\))) to understand the underlying pattern.
1.2.2 Coping strategies
1.2.2.1 Step 1: LCA model selection
Build and compare LCA models
| Model | G2 | AIC | BIC |
|---|---|---|---|
| 2 | 29.066630 | 4548.577 | 4596.402 |
| 3 | 6.162343 | 4535.673 | 4610.067 |
| 4 | 6.058262 | 4545.569 | 4646.533 |
| 5 | 1.878940 | 4551.389 | 4678.923 |
1.2.2.2 Step 2: Classification error: averaged posterior probability(APP)
print averaged posterior probability(APP) for coping strategy, results suggest low classification error(APP>0.7).
## [1] 0.8685438 0.7917650 0.9089532
1.2.2.3 Step 3: Final latent class probabilities
Similarly, get the predicted posterior probability of the latent variable for coping strategy.
1.2.2.4 Step 4: Assign label to each latent class (coping strategies)
1.3 Missing values
98.6% of the subjects have complete information.
Tabulate the number and percentage of missing. The missing rate is low, so complete case analysis will be used later.
Check if there is any missing pattern among missing variables: No pattern presents!
1.4 The final working dataset
After some investigation, I decided to do a complete case analysis(CCA), the final working dataset is then generated.
2 Descriptive table (table 1)
A Descriptive table is generated using the 1441 subjects with complete data. Summary stats is stratified by treatment condition.
| Demographics | Treatment Received | ||
|---|---|---|---|
| Placebo Control N = 4881 |
Project ABC N = 4891 |
Project Personality N = 4641 |
|
| Baseline CDI mean score(0-2) | 1.16 (0.35) | 1.15 (0.34) | 1.17 (0.36) |
| Race | |||
| Asian Including Asian Desi | 50 (10%) | 58 (12%) | 50 (11%) |
| Black/African-American | 33 (6.8%) | 40 (8.2%) | 36 (7.8%) |
| Hispanic/Latinx | 57 (12%) | 61 (12%) | 53 (11%) |
| Mixed | 74 (15%) | 68 (14%) | 63 (14%) |
| White | 274 (56%) | 262 (54%) | 262 (56%) |
| Age (yrs) | |||
| 13 | 28 (5.7%) | 32 (6.5%) | 28 (6.0%) |
| 14 | 77 (16%) | 81 (17%) | 63 (14%) |
| 15 | 150 (31%) | 156 (32%) | 162 (35%) |
| 16 | 233 (48%) | 220 (45%) | 211 (45%) |
| Biological sex | |||
| Female | 434 (89%) | 437 (89%) | 418 (90%) |
| Male | 54 (11%) | 52 (11%) | 46 (9.9%) |
| Sexual orientation | |||
| Heterosexual | 108 (22%) | 97 (20%) | 106 (23%) |
| LGBTQ | 309 (63%) | 327 (67%) | 291 (63%) |
| Other | 71 (15%) | 65 (13%) | 67 (14%) |
| Language | |||
| English | 476 (98%) | 476 (97%) | 450 (97%) |
| Other | 12 (2.5%) | 13 (2.7%) | 14 (3.0%) |
| Gender identity | |||
| Non-binary | 102 (21%) | 98 (20%) | 86 (19%) |
| Women/girls | 313 (64%) | 325 (66%) | 306 (66%) |
| Male/Masculine | 73 (15%) | 66 (13%) | 72 (16%) |
| Number of challenges | |||
| 0 | 98 (20%) | 99 (20%) | 85 (18%) |
| 1 | 276 (57%) | 297 (61%) | 275 (59%) |
| >=2 | 114 (23%) | 93 (19%) | 104 (22%) |
| Type of challenges | |||
| No impact | 98 (20%) | 99 (20%) | 85 (18%) |
| Other | 119 (24%) | 97 (20%) | 124 (27%) |
| School | 271 (56%) | 293 (60%) | 255 (55%) |
| Type of coping strategies | |||
| Combined | 58 (12%) | 61 (12%) | 53 (11%) |
| No action | 228 (47%) | 235 (48%) | 236 (51%) |
| Positive | 202 (41%) | 193 (39%) | 175 (38%) |
| 1 Mean (SD); n (%) | |||
## % latex table generated in R 4.3.1 by xtable 1.8-4 package
## % Fri Aug 2 17:38:28 2024
## \begin{table}[ht]
## \centering
## \begin{tabular}{llll}
## \hline
## **Demographics** & **Placebo Control**
## N = 488 & **Project ABC**
## N = 489 & **Project Personality**
## N = 464 \\
## \hline
## Baseline CDI mean score(0-2) & 1.16 (0.35) & 1.15 (0.34) & 1.17 (0.36) \\
## Race & & & \\
## Asian Including Asian Desi & 50 (10\%) & 58 (12\%) & 50 (11\%) \\
## Black/African-American & 33 (6.8\%) & 40 (8.2\%) & 36 (7.8\%) \\
## Hispanic/Latinx & 57 (12\%) & 61 (12\%) & 53 (11\%) \\
## Mixed & 74 (15\%) & 68 (14\%) & 63 (14\%) \\
## White & 274 (56\%) & 262 (54\%) & 262 (56\%) \\
## Age (yrs) & & & \\
## 13 & 28 (5.7\%) & 32 (6.5\%) & 28 (6.0\%) \\
## 14 & 77 (16\%) & 81 (17\%) & 63 (14\%) \\
## 15 & 150 (31\%) & 156 (32\%) & 162 (35\%) \\
## 16 & 233 (48\%) & 220 (45\%) & 211 (45\%) \\
## Biological sex & & & \\
## Female & 434 (89\%) & 437 (89\%) & 418 (90\%) \\
## Male & 54 (11\%) & 52 (11\%) & 46 (9.9\%) \\
## Sexual orientation & & & \\
## Heterosexual & 108 (22\%) & 97 (20\%) & 106 (23\%) \\
## LGBTQ & 309 (63\%) & 327 (67\%) & 291 (63\%) \\
## Other & 71 (15\%) & 65 (13\%) & 67 (14\%) \\
## Language & & & \\
## English & 476 (98\%) & 476 (97\%) & 450 (97\%) \\
## Other & 12 (2.5\%) & 13 (2.7\%) & 14 (3.0\%) \\
## Gender identity & & & \\
## Non-binary & 102 (21\%) & 98 (20\%) & 86 (19\%) \\
## Women/girls & 313 (64\%) & 325 (66\%) & 306 (66\%) \\
## Male/Masculine & 73 (15\%) & 66 (13\%) & 72 (16\%) \\
## Number of challenges & & & \\
## 0 & 98 (20\%) & 99 (20\%) & 85 (18\%) \\
## 1 & 276 (57\%) & 297 (61\%) & 275 (59\%) \\
## $>$=2 & 114 (23\%) & 93 (19\%) & 104 (22\%) \\
## Type of challenges & & & \\
## No impact & 98 (20\%) & 99 (20\%) & 85 (18\%) \\
## Other & 119 (24\%) & 97 (20\%) & 124 (27\%) \\
## School & 271 (56\%) & 293 (60\%) & 255 (55\%) \\
## Type of coping strategies & & & \\
## Combined & 58 (12\%) & 61 (12\%) & 53 (11\%) \\
## No action & 228 (47\%) & 235 (48\%) & 236 (51\%) \\
## Positive & 202 (41\%) & 193 (39\%) & 175 (38\%) \\
## \hline
## \end{tabular}
## \end{table}
3 Baseline CDI Prediction Model
Will use risk-based approach.
The first step is to construct the baseline CDI prediction model. The focus of the “risk” prediction model is on accurately predicting individuals’ “disease” risk. Several considerations need to be taken into account:
Statistical Model Selection:
Consideration of models: will use linear model (simple and interpretability)
Performance Metrics:
For continuous outcomes, models will be evaluated and selected based on root mean squared error (RMSE), calibration slope, and calibration-at-large.
Overfitting:
Employ a leave-one-out cross-validation (LOOCV) framework to address overfitting. LOOCV will be conducted on 80% of the samples (derivation cohort), while the remaining 20% will serve as a test cohort to mimic an external validation. Final model will be constructed on the entire dataset.
a glimpse of the variables:
## tibble [977 × 11] (S3: tbl_df/tbl/data.frame)
## $ b_response_id : chr [1:977] "R_02uyQw7i0v8yQRX" "R_089oJuXofDsITq9" "R_08lyOxzUAzbJqcV" "R_0lDh6r7xrSRAemR" ...
## $ condition : Factor w/ 2 levels "Placebo Control",..: 1 1 1 2 1 1 1 2 2 2 ...
## $ b_cdi_mean : num [1:977] 1.417 1.167 1.333 1 0.917 ...
## $ f1_cdi_mean : num [1:977] 1 0.583 1.417 0.833 0.583 ...
## $ recode_race : Factor w/ 5 levels "Asian Including Asian Desi",..: 5 5 5 1 5 4 5 5 5 5 ...
## $ b_screener_age: num [1:977] 14 16 15 14 16 14 14 16 16 16 ...
## $ b_dem_sex : Factor w/ 2 levels "Female","Male": 1 1 1 1 1 1 1 1 1 1 ...
## $ recode_orient : Factor w/ 3 levels "Heterosexual",..: 2 2 2 1 1 2 1 1 2 2 ...
## $ genderid3 : Factor w/ 3 levels "1","2","3": 1 2 2 2 2 2 2 2 2 2 ...
## $ family_cat : Factor w/ 3 levels "No impact","Other",..: 2 1 3 3 3 2 3 2 3 2 ...
## $ cope3 : Factor w/ 3 levels "1","2","3": 3 3 2 2 2 2 2 2 2 2 ...
3.1 Build prediction model
Build prediction models using caret
Summary of baseline model output of project ABC model:
## Length Class Mode
## a0 100 -none- numeric
## beta 1300 dgCMatrix S4
## df 100 -none- numeric
## dim 2 -none- numeric
## lambda 100 -none- numeric
## dev.ratio 100 -none- numeric
## nulldev 1 -none- numeric
## npasses 1 -none- numeric
## jerr 1 -none- numeric
## offset 1 -none- logical
## call 5 -none- call
## nobs 1 -none- numeric
## lambdaOpt 1 -none- numeric
## xNames 13 -none- character
## problemType 1 -none- character
## tuneValue 2 data.frame list
## obsLevels 1 -none- logical
## param 0 -none- list
Summary of baseline model output of project personality model:
## Length Class Mode
## a0 65 -none- numeric
## beta 845 dgCMatrix S4
## df 65 -none- numeric
## dim 2 -none- numeric
## lambda 65 -none- numeric
## dev.ratio 65 -none- numeric
## nulldev 1 -none- numeric
## npasses 1 -none- numeric
## jerr 1 -none- numeric
## offset 1 -none- logical
## call 5 -none- call
## nobs 1 -none- numeric
## lambdaOpt 1 -none- numeric
## xNames 13 -none- character
## problemType 1 -none- character
## tuneValue 2 data.frame list
## obsLevels 1 -none- logical
## param 0 -none- list
3.2 Evaluate model performance
Summaries the model performance in the following table:
| Metric | Validation | Test |
|---|---|---|
| RMSE | 0.3992314 | 0.3959155 |
| Calibration Slope | 1.1575945 | 0.9232901 |
| Calibration at Large | -0.1543406 | 0.0670571 |
| Metric | Validation | Test |
|---|---|---|
| RMSE | 0.4055562 | 0.4266310 |
| Calibration Slope | 1.0819270 | 0.0040360 |
| Calibration at Large | -0.0809549 | 0.9874272 |
3.3 Final baseline prediciton model
The final model is constructed on the entire dataset
## 14 x 1 sparse Matrix of class "dgCMatrix"
## s0
## (Intercept) 1.024079643
## recode_raceBlack/African-American -0.029861721
## recode_raceHispanic/Latinx -0.067205445
## recode_raceMixed -0.008337803
## recode_raceWhite 0.041071994
## b_dem_sexMale -0.121873161
## recode_orientLGBTQ 0.073487928
## recode_orientOther 0.102386860
## genderid32 -0.109173072
## genderid33 -0.018497319
## family_catOther 0.060132421
## family_catSchool 0.020780601
## cope32 -0.024043888
## cope33 -0.112895337
## 14 x 1 sparse Matrix of class "dgCMatrix"
## s0
## (Intercept) 1.152897654
## recode_raceBlack/African-American -0.054500296
## recode_raceHispanic/Latinx -0.048753462
## recode_raceMixed -0.030923716
## recode_raceWhite -0.002056701
## b_dem_sexMale -0.104694777
## recode_orientLGBTQ 0.068796666
## recode_orientOther 0.064098555
## genderid32 -0.146439033
## genderid33 -0.102595869
## family_catOther 0.020356363
## family_catSchool -0.015311446
## cope32 -0.054863363
## cope33 -0.125546490
3.3.1 Project ABC baseline model
A summary of the baseline prediction model, for project ABC:
| Dependent variable | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 1.06 | 0.92 – 1.21 | <0.001 |
|
recode race [Black/African-American] |
-0.04 | -0.16 – 0.08 | 0.527 |
|
recode race [Hispanic/Latinx] |
-0.08 | -0.19 – 0.02 | 0.129 |
| recode race [Mixed] | -0.03 | -0.13 – 0.08 | 0.617 |
| recode race [White] | 0.03 | -0.05 – 0.12 | 0.462 |
| b dem sex [Male] | -0.15 | -0.25 – -0.04 | 0.007 |
| recode orient [LGBTQ] | 0.09 | 0.02 – 0.16 | 0.010 |
| recode orient [Other] | 0.12 | 0.03 – 0.22 | 0.008 |
| genderid3 [2] | -0.13 | -0.20 – -0.06 | <0.001 |
| genderid3 [3] | -0.02 | -0.12 – 0.08 | 0.687 |
| family cat [Other] | 0.07 | -0.01 – 0.15 | 0.070 |
| family cat [School] | 0.03 | -0.03 – 0.10 | 0.313 |
| cope3 [2] | -0.06 | -0.14 – 0.02 | 0.137 |
| cope3 [3] | -0.16 | -0.24 – -0.08 | <0.001 |
| Observations | 977 | ||
| R2 / R2 adjusted | 0.079 / 0.067 | ||
3.3.2 Project personality baseline model
for project personality:| Dependent variable | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 1.16 | 1.00 – 1.31 | <0.001 |
|
recode race [Black/African-American] |
-0.06 | -0.18 – 0.07 | 0.387 |
|
recode race [Hispanic/Latinx] |
-0.05 | -0.16 – 0.06 | 0.382 |
| recode race [Mixed] | -0.03 | -0.14 – 0.08 | 0.552 |
| recode race [White] | -0.00 | -0.09 – 0.09 | 0.937 |
| b dem sex [Male] | -0.11 | -0.22 – 0.01 | 0.066 |
| recode orient [LGBTQ] | 0.07 | 0.00 – 0.14 | 0.046 |
| recode orient [Other] | 0.06 | -0.03 – 0.16 | 0.182 |
| genderid3 [2] | -0.15 | -0.22 – -0.08 | <0.001 |
| genderid3 [3] | -0.10 | -0.21 – -0.00 | 0.049 |
| family cat [Other] | 0.02 | -0.06 – 0.10 | 0.625 |
| family cat [School] | -0.02 | -0.09 – 0.05 | 0.664 |
| cope3 [2] | -0.06 | -0.15 – 0.03 | 0.194 |
| cope3 [3] | -0.13 | -0.22 – -0.04 | 0.005 |
| Observations | 952 | ||
| R2 / R2 adjusted | 0.052 / 0.039 | ||
3.4 LRT
LRT was used to test the sig. of variables in the baseline model
For project ABC:
| Variable | Deviance | DF | P_value |
|---|---|---|---|
| Gender identity | 15.456260 | 2 | 0.00 |
| Race | 9.522943 | 4 | 0.05 |
| Biological sex | 7.509756 | 1 | 0.01 |
| Sexual Orientation | 8.728432 | 2 | 0.01 |
| Challenges | 3.339847 | 2 | 0.19 |
| Coping strategies | 19.418374 | 2 | 0.00 |
For project personality
| Variable | Deviance | DF | P_value |
|---|---|---|---|
| Gender identity | 16.358782 | 2 | 0.00 |
| Race | 2.266644 | 4 | 0.69 |
| Biological sex | 3.441522 | 1 | 0.06 |
| Sexual Orientation | 4.084130 | 2 | 0.13 |
| Challenges | 1.220614 | 2 | 0.54 |
| Coping strategies | 10.608348 | 2 | 0.00 |
For Project Personality
4 Investigate HTE
4.1 Main effect only model
first, replicate the model in the original paper for comparison. the main effect model which adjusted for baseline CDI score is specified as:\[E(Y|\bf{X}) = \text{baseline CDI}+ \text{condition}\]
print the model, compare with the published paper (results are similar): for project ABC vs control:
##
## Call:
## lm(formula = f1_cdi_mean ~ b_cdi_mean + condition, data = abc_main_dat)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.18567 -0.21921 0.01075 0.22509 1.04660
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.27471 0.04097 6.705 3.42e-11 ***
## b_cdi_mean 0.64303 0.03263 19.706 < 2e-16 ***
## conditionProject ABC -0.07953 0.02234 -3.560 0.000389 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3492 on 974 degrees of freedom
## Multiple R-squared: 0.2922, Adjusted R-squared: 0.2907
## F-statistic: 201 on 2 and 974 DF, p-value: < 2.2e-16
| f 1 cdi mean | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 0.27 | 0.19 – 0.36 | <0.001 |
| b cdi mean | 0.64 | 0.58 – 0.71 | <0.001 |
| condition [Project ABC] | -0.08 | -0.12 – -0.04 | <0.001 |
| Observations | 977 | ||
| R2 / R2 adjusted | 0.292 / 0.291 | ||
for project personality vs control:
##
## Call:
## lm(formula = f1_cdi_mean ~ b_cdi_mean + condition, data = person_main_dat)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.17752 -0.22054 0.01222 0.23434 1.03109
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.31125 0.04188 7.432 2.38e-13 ***
## b_cdi_mean 0.61148 0.03329 18.366 < 2e-16 ***
## conditionProject Personality -0.06895 0.02337 -2.950 0.00325 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3604 on 949 degrees of freedom
## Multiple R-squared: 0.2664, Adjusted R-squared: 0.2649
## F-statistic: 172.3 on 2 and 949 DF, p-value: < 2.2e-16
| f 1 cdi mean | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 0.31 | 0.23 – 0.39 | <0.001 |
| b cdi mean | 0.61 | 0.55 – 0.68 | <0.001 |
|
condition [Project Personality] |
-0.07 | -0.11 – -0.02 | 0.003 |
| Observations | 952 | ||
| R2 / R2 adjusted | 0.266 / 0.265 | ||
4.2 HTE model
The HTE is defined as: \[E(Y|\bf{X}) = \text{baseline CDI}+ \text{condition} + \text{lp}+\text{condition}\times \text{lp}\] where lp is the linear predictor of the baseline model.
4.2.1 HTE Project ABC
The hte model for Project ABC vs control:
##
## Call:
## lm(formula = f1_cdi_mean ~ b_cdi_mean + condition * lp_abc, data = dat_abc)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.16587 -0.21245 0.01587 0.22319 1.04761
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.20672 0.13296 1.555 0.1203
## b_cdi_mean 0.60042 0.03517 17.070 <2e-16 ***
## conditionProject ABC -0.46326 0.18742 -2.472 0.0136 *
## lp_abc 0.11985 0.14022 0.855 0.3929
## conditionProject ABC:lp_abc 0.39263 0.19023 2.064 0.0393 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3471 on 972 degrees of freedom
## Multiple R-squared: 0.302, Adjusted R-squared: 0.2991
## F-statistic: 105.1 on 4 and 972 DF, p-value: < 2.2e-16
| f 1 cdi mean | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 0.21 | -0.05 – 0.47 | 0.120 |
| b cdi mean | 0.60 | 0.53 – 0.67 | <0.001 |
| condition [Project ABC] | -0.46 | -0.83 – -0.10 | 0.014 |
| lp abc | 0.12 | -0.16 – 0.40 | 0.393 |
|
condition [Project ABC] × lp abc |
0.39 | 0.02 – 0.77 | 0.039 |
| Observations | 977 | ||
| R2 / R2 adjusted | 0.302 / 0.299 | ||
4.2.2 HTE Project personality
The hte model for Project personality vs control:
##
## Call:
## lm(formula = f1_cdi_mean ~ b_cdi_mean + condition * lp_person,
## data = dat_person)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.1604 -0.2189 0.0176 0.2321 1.0427
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.18881 0.16684 1.132 0.258
## b_cdi_mean 0.58760 0.03544 16.580 <2e-16 ***
## conditionProject Personality -0.28385 0.24270 -1.170 0.242
## lp_person 0.15184 0.17399 0.873 0.383
## conditionProject Personality:lp_person 0.21744 0.24431 0.890 0.374
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3599 on 947 degrees of freedom
## Multiple R-squared: 0.27, Adjusted R-squared: 0.2669
## F-statistic: 87.56 on 4 and 947 DF, p-value: < 2.2e-16
| f 1 cdi mean | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 0.19 | -0.14 – 0.52 | 0.258 |
| b cdi mean | 0.59 | 0.52 – 0.66 | <0.001 |
|
condition [Project Personality] |
-0.28 | -0.76 – 0.19 | 0.242 |
| lp person | 0.15 | -0.19 – 0.49 | 0.383 |
|
condition [Project Personality] × lp person |
0.22 | -0.26 – 0.70 | 0.374 |
| Observations | 952 | ||
| R2 / R2 adjusted | 0.270 / 0.267 | ||
5 Evaluate HTE
The HTE is evaluated using calibration plot.
5.1 Point estiamtes
First, report the point estiamtes of ATE and cATE.
| Comparsion | ATE | SE | |
|---|---|---|---|
| conditionProject ABC | Project ABC vs Control | -0.0795347 | 0.0223413 |
| conditionProject Personality | Project personality vs Control | -0.0689513 | 0.0233696 |
A brief summary of the linear predictors:
## Length Class Mode
## 0 NULL NULL
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.7486 0.9316 0.9751 0.9888 1.0464 1.2441
5.2 Bootstrapped CI
compute bootstrapped CI for cATE:
for Project personality:
5.3 Calibration plots
The linear predictor entered model as a continuous variable. For presentation purpose, the linear predictor is discretized into five “risk” groups using quantiles (0.2 incremental). The averaged HTEs/cATEs by “risk” group are calculated and compared with the ATE.
For Project ABC vs control:
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.6620 0.8980 0.9743 0.9782 1.0605 1.2931
For Project personality vs control:
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.7486 0.9316 0.9751 0.9888 1.0464 1.2441
combined plots for reporting purpose:
5.3.1 Rearrange for report (For Porject ABC vs control)
Rearrange tables for reporting purpose:
## Joining with `by = join_by(quantile_grp)`
The calibration table and plot:
5.3.2 Rearrange for report (For Porject personality vs control)
## Joining with `by = join_by(quantile_grp)`
The calibration table and plot:
5.4 Distribution in high risk groups
## % latex table generated in R 4.3.1 by xtable 1.8-4 package
## % Fri Aug 2 17:40:22 2024
## \begin{table}[ht]
## \centering
## \begin{tabular}{lllllllllll}
## \hline
## **Demographics** & **1**
## N = 98 & **10**
## N = 94 & **2**
## N = 104 & **3**
## N = 92 & **4**
## N = 97 & **5**
## N = 101 & **6**
## N = 95 & **7**
## N = 103 & **8**
## N = 91 & **9**
## N = 102 \\
## \hline
## condition & & & & & & & & & & \\
## Placebo Control & 44 (45\%) & 42 (45\%) & 57 (55\%) & 52 (57\%) & 51 (53\%) & 44 (44\%) & 41 (43\%) & 51 (50\%) & 45 (49\%) & 61 (60\%) \\
## Project ABC & 54 (55\%) & 52 (55\%) & 47 (45\%) & 40 (43\%) & 46 (47\%) & 57 (56\%) & 54 (57\%) & 52 (50\%) & 46 (51\%) & 41 (40\%) \\
## Baseline CDI mean score(0-2) & 0.92 (0.31) & 1.37 (0.30) & 1.04 (0.37) & 1.06 (0.33) & 1.05 (0.29) & 1.14 (0.34) & 1.23 (0.31) & 1.21 (0.30) & 1.26 (0.32) & 1.29 (0.30) \\
## Race & & & & & & & & & & \\
## Asian Including Asian Desi & 18 (18\%) & 1 (1.1\%) & 13 (13\%) & 35 (38\%) & 3 (3.1\%) & 9 (8.9\%) & 10 (11\%) & 9 (8.7\%) & 5 (5.5\%) & 5 (4.9\%) \\
## Black/African-American & 16 (16\%) & 4 (4.3\%) & 16 (15\%) & 5 (5.4\%) & 5 (5.2\%) & 11 (11\%) & 8 (8.4\%) & 3 (2.9\%) & 3 (3.3\%) & 2 (2.0\%) \\
## Hispanic/Latinx & 38 (39\%) & 1 (1.1\%) & 27 (26\%) & 1 (1.1\%) & 16 (16\%) & 20 (20\%) & 2 (2.1\%) & 3 (2.9\%) & 5 (5.5\%) & 5 (4.9\%) \\
## Mixed & 10 (10\%) & 14 (15\%) & 26 (25\%) & 1 (1.1\%) & 13 (13\%) & 17 (17\%) & 21 (22\%) & 11 (11\%) & 12 (13\%) & 17 (17\%) \\
## White & 16 (16\%) & 74 (79\%) & 22 (21\%) & 50 (54\%) & 60 (62\%) & 44 (44\%) & 54 (57\%) & 77 (75\%) & 66 (73\%) & 73 (72\%) \\
## Age (yrs) & & & & & & & & & & \\
## 13 & 3 (3.1\%) & 10 (11\%) & 4 (3.8\%) & 1 (1.1\%) & 5 (5.2\%) & 6 (5.9\%) & 7 (7.4\%) & 8 (7.8\%) & 9 (9.9\%) & 7 (6.9\%) \\
## 14 & 12 (12\%) & 16 (17\%) & 13 (13\%) & 19 (21\%) & 11 (11\%) & 21 (21\%) & 15 (16\%) & 12 (12\%) & 20 (22\%) & 19 (19\%) \\
## 15 & 32 (33\%) & 30 (32\%) & 30 (29\%) & 29 (32\%) & 27 (28\%) & 28 (28\%) & 27 (28\%) & 39 (38\%) & 27 (30\%) & 37 (36\%) \\
## 16 & 51 (52\%) & 38 (40\%) & 57 (55\%) & 43 (47\%) & 54 (56\%) & 46 (46\%) & 46 (48\%) & 44 (43\%) & 35 (38\%) & 39 (38\%) \\
## Biological sex & & & & & & & & & & \\
## Female & 66 (67\%) & 94 (100\%) & 83 (80\%) & 80 (87\%) & 87 (90\%) & 92 (91\%) & 80 (84\%) & 101 (98\%) & 87 (96\%) & 101 (99\%) \\
## Male & 32 (33\%) & 0 (0\%) & 21 (20\%) & 12 (13\%) & 10 (10\%) & 9 (8.9\%) & 15 (16\%) & 2 (1.9\%) & 4 (4.4\%) & 1 (1.0\%) \\
## Sexual orientation & & & & & & & & & & \\
## Heterosexual & 77 (79\%) & 0 (0\%) & 45 (43\%) & 39 (42\%) & 5 (5.2\%) & 20 (20\%) & 16 (17\%) & 0 (0\%) & 3 (3.3\%) & 0 (0\%) \\
## LGBTQ & 20 (20\%) & 64 (68\%) & 57 (55\%) & 48 (52\%) & 83 (86\%) & 68 (67\%) & 75 (79\%) & 90 (87\%) & 54 (59\%) & 77 (75\%) \\
## Other & 1 (1.0\%) & 30 (32\%) & 2 (1.9\%) & 5 (5.4\%) & 9 (9.3\%) & 13 (13\%) & 4 (4.2\%) & 13 (13\%) & 34 (37\%) & 25 (25\%) \\
## Language & & & & & & & & & & \\
## English & 91 (93\%) & 93 (99\%) & 100 (96\%) & 88 (96\%) & 94 (97\%) & 99 (98\%) & 94 (99\%) & 102 (99\%) & 91 (100\%) & 100 (98\%) \\
## Other & 7 (7.1\%) & 1 (1.1\%) & 4 (3.8\%) & 4 (4.3\%) & 3 (3.1\%) & 2 (2.0\%) & 1 (1.1\%) & 1 (1.0\%) & 0 (0\%) & 2 (2.0\%) \\
## Gender identity & & & & & & & & & & \\
## Non-binary & 1 (1.0\%) & 79 (84\%) & 2 (1.9\%) & 1 (1.1\%) & 5 (5.2\%) & 5 (5.0\%) & 20 (21\%) & 10 (9.7\%) & 27 (30\%) & 50 (49\%) \\
## Women/girls & 70 (71\%) & 0 (0\%) & 83 (80\%) & 80 (87\%) & 86 (89\%) & 88 (87\%) & 61 (64\%) & 87 (84\%) & 48 (53\%) & 35 (34\%) \\
## Male/Masculine & 27 (28\%) & 15 (16\%) & 19 (18\%) & 11 (12\%) & 6 (6.2\%) & 8 (7.9\%) & 14 (15\%) & 6 (5.8\%) & 16 (18\%) & 17 (17\%) \\
## Number of challenges & & & & & & & & & & \\
## 0 & 26 (27\%) & 8 (8.5\%) & 31 (30\%) & 37 (40\%) & 9 (9.3\%) & 9 (8.9\%) & 30 (32\%) & 16 (16\%) & 17 (19\%) & 14 (14\%) \\
## 1 & 58 (59\%) & 44 (47\%) & 65 (63\%) & 46 (50\%) & 77 (79\%) & 58 (57\%) & 44 (46\%) & 74 (72\%) & 47 (52\%) & 60 (59\%) \\
## $>$=2 & 14 (14\%) & 42 (45\%) & 8 (7.7\%) & 9 (9.8\%) & 11 (11\%) & 34 (34\%) & 21 (22\%) & 13 (13\%) & 27 (30\%) & 28 (27\%) \\
## Type of challenges & & & & & & & & & & \\
## No impact & 26 (27\%) & 8 (8.5\%) & 31 (30\%) & 37 (40\%) & 9 (9.3\%) & 9 (8.9\%) & 30 (32\%) & 16 (16\%) & 17 (19\%) & 14 (14\%) \\
## Other & 11 (11\%) & 45 (48\%) & 10 (9.6\%) & 12 (13\%) & 9 (9.3\%) & 33 (33\%) & 27 (28\%) & 6 (5.8\%) & 25 (27\%) & 38 (37\%) \\
## School & 61 (62\%) & 41 (44\%) & 63 (61\%) & 43 (47\%) & 79 (81\%) & 59 (58\%) & 38 (40\%) & 81 (79\%) & 49 (54\%) & 50 (49\%) \\
## Type of coping strategies & & & & & & & & & & \\
## Combined & 0 (0\%) & 35 (37\%) & 0 (0\%) & 0 (0\%) & 2 (2.1\%) & 7 (6.9\%) & 9 (9.5\%) & 13 (13\%) & 17 (19\%) & 36 (35\%) \\
## No action & 11 (11\%) & 59 (63\%) & 39 (38\%) & 33 (36\%) & 25 (26\%) & 62 (61\%) & 67 (71\%) & 78 (76\%) & 47 (52\%) & 42 (41\%) \\
## Positive & 87 (89\%) & 0 (0\%) & 65 (63\%) & 59 (64\%) & 70 (72\%) & 32 (32\%) & 19 (20\%) & 12 (12\%) & 27 (30\%) & 24 (24\%) \\
## abc\_ate & & & & & & & & & & \\
## -0.0795346966023499 & 98 (100\%) & 94 (100\%) & 104 (100\%) & 92 (100\%) & 97 (100\%) & 101 (100\%) & 95 (100\%) & 103 (100\%) & 91 (100\%) & 102 (100\%) \\
## abc\_hte & -0.16 (0.01) & 0.00 (0.02) & -0.13 (0.01) & -0.11 (0.00) & -0.10 (0.00) & -0.09 (0.00) & -0.07 (0.00) & -0.06 (0.00) & -0.05 (0.00) & -0.03 (0.01) \\
## lp\_abc & 0.78 (0.03) & 1.19 (0.04) & 0.86 (0.02) & 0.90 (0.01) & 0.92 (0.01) & 0.96 (0.01) & 1.00 (0.01) & 1.03 (0.00) & 1.06 (0.01) & 1.11 (0.02) \\
## \hline
## \end{tabular}
## \end{table}
## % latex table generated in R 4.3.1 by xtable 1.8-4 package
## % Fri Aug 2 17:40:24 2024
## \begin{table}[ht]
## \centering
## \begin{tabular}{lllllllllll}
## \hline
## **Demographics** & **1**
## N = 101 & **10**
## N = 96 & **2**
## N = 94 & **3**
## N = 103 & **4**
## N = 83 & **5**
## N = 95 & **6**
## N = 120 & **7**
## N = 72 & **8**
## N = 94 & **9**
## N = 94 \\
## \hline
## condition & & & & & & & & & & \\
## Placebo Control & 56 (55\%) & 46 (48\%) & 53 (56\%) & 46 (45\%) & 44 (53\%) & 47 (49\%) & 63 (53\%) & 33 (46\%) & 46 (49\%) & 54 (57\%) \\
## Project Personality & 45 (45\%) & 50 (52\%) & 41 (44\%) & 57 (55\%) & 39 (47\%) & 48 (51\%) & 57 (48\%) & 39 (54\%) & 48 (51\%) & 40 (43\%) \\
## Baseline CDI mean score(0-2) & 0.93 (0.32) & 1.35 (0.30) & 1.02 (0.39) & 1.06 (0.33) & 1.12 (0.37) & 1.19 (0.31) & 1.19 (0.28) & 1.25 (0.30) & 1.24 (0.36) & 1.29 (0.33) \\
## Race & & & & & & & & & & \\
## Asian Including Asian Desi & 11 (11\%) & 6 (6.3\%) & 9 (9.6\%) & 3 (2.9\%) & 26 (31\%) & 15 (16\%) & 5 (4.2\%) & 15 (21\%) & 2 (2.1\%) & 8 (8.5\%) \\
## Black/African-American & 14 (14\%) & 3 (3.1\%) & 12 (13\%) & 12 (12\%) & 8 (9.6\%) & 5 (5.3\%) & 4 (3.3\%) & 3 (4.2\%) & 2 (2.1\%) & 6 (6.4\%) \\
## Hispanic/Latinx & 20 (20\%) & 3 (3.1\%) & 26 (28\%) & 8 (7.8\%) & 0 (0\%) & 24 (25\%) & 13 (11\%) & 5 (6.9\%) & 3 (3.2\%) & 8 (8.5\%) \\
## Mixed & 9 (8.9\%) & 14 (15\%) & 23 (24\%) & 10 (9.7\%) & 9 (11\%) & 12 (13\%) & 9 (7.5\%) & 19 (26\%) & 17 (18\%) & 15 (16\%) \\
## White & 47 (47\%) & 70 (73\%) & 24 (26\%) & 70 (68\%) & 40 (48\%) & 39 (41\%) & 89 (74\%) & 30 (42\%) & 70 (74\%) & 57 (61\%) \\
## Age (yrs) & & & & & & & & & & \\
## 13 & 3 (3.0\%) & 7 (7.3\%) & 4 (4.3\%) & 1 (1.0\%) & 3 (3.6\%) & 10 (11\%) & 9 (7.5\%) & 7 (9.7\%) & 7 (7.4\%) & 5 (5.3\%) \\
## 14 & 15 (15\%) & 17 (18\%) & 11 (12\%) & 18 (17\%) & 15 (18\%) & 9 (9.5\%) & 15 (13\%) & 13 (18\%) & 13 (14\%) & 14 (15\%) \\
## 15 & 39 (39\%) & 28 (29\%) & 28 (30\%) & 34 (33\%) & 25 (30\%) & 31 (33\%) & 37 (31\%) & 23 (32\%) & 32 (34\%) & 35 (37\%) \\
## 16 & 44 (44\%) & 44 (46\%) & 51 (54\%) & 50 (49\%) & 40 (48\%) & 45 (47\%) & 59 (49\%) & 29 (40\%) & 42 (45\%) & 40 (43\%) \\
## Biological sex & & & & & & & & & & \\
## Female & 58 (57\%) & 96 (100\%) & 77 (82\%) & 94 (91\%) & 78 (94\%) & 87 (92\%) & 113 (94\%) & 71 (99\%) & 87 (93\%) & 91 (97\%) \\
## Male & 43 (43\%) & 0 (0\%) & 17 (18\%) & 9 (8.7\%) & 5 (6.0\%) & 8 (8.4\%) & 7 (5.8\%) & 1 (1.4\%) & 7 (7.4\%) & 3 (3.2\%) \\
## Sexual orientation & & & & & & & & & & \\
## Heterosexual & 86 (85\%) & 0 (0\%) & 44 (47\%) & 29 (28\%) & 27 (33\%) & 18 (19\%) & 7 (5.8\%) & 3 (4.2\%) & 0 (0\%) & 0 (0\%) \\
## LGBTQ & 13 (13\%) & 66 (69\%) & 45 (48\%) & 66 (64\%) & 51 (61\%) & 63 (66\%) & 87 (73\%) & 58 (81\%) & 76 (81\%) & 75 (80\%) \\
## Other & 2 (2.0\%) & 30 (31\%) & 5 (5.3\%) & 8 (7.8\%) & 5 (6.0\%) & 14 (15\%) & 26 (22\%) & 11 (15\%) & 18 (19\%) & 19 (20\%) \\
## Language & & & & & & & & & & \\
## English & 94 (93\%) & 94 (98\%) & 90 (96\%) & 100 (97\%) & 80 (96\%) & 92 (97\%) & 119 (99\%) & 70 (97\%) & 94 (100\%) & 93 (99\%) \\
## Other & 7 (6.9\%) & 2 (2.1\%) & 4 (4.3\%) & 3 (2.9\%) & 3 (3.6\%) & 3 (3.2\%) & 1 (0.8\%) & 2 (2.8\%) & 0 (0\%) & 1 (1.1\%) \\
## Gender identity & & & & & & & & & & \\
## Non-binary & 0 (0\%) & 91 (95\%) & 0 (0\%) & 2 (1.9\%) & 0 (0\%) & 2 (2.1\%) & 3 (2.5\%) & 3 (4.2\%) & 20 (21\%) & 67 (71\%) \\
## Women/girls & 61 (60\%) & 0 (0\%) & 77 (82\%) & 93 (90\%) & 78 (94\%) & 82 (86\%) & 95 (79\%) & 61 (85\%) & 55 (59\%) & 17 (18\%) \\
## Male/Masculine & 40 (40\%) & 5 (5.2\%) & 17 (18\%) & 8 (7.8\%) & 5 (6.0\%) & 11 (12\%) & 22 (18\%) & 8 (11\%) & 19 (20\%) & 10 (11\%) \\
## Number of challenges & & & & & & & & & & \\
## 0 & 10 (9.9\%) & 20 (21\%) & 27 (29\%) & 4 (3.9\%) & 37 (45\%) & 20 (21\%) & 15 (13\%) & 25 (35\%) & 11 (12\%) & 14 (15\%) \\
## 1 & 78 (77\%) & 39 (41\%) & 56 (60\%) & 77 (75\%) & 37 (45\%) & 48 (51\%) & 85 (71\%) & 30 (42\%) & 58 (62\%) & 43 (46\%) \\
## $>$=2 & 13 (13\%) & 37 (39\%) & 11 (12\%) & 22 (21\%) & 9 (11\%) & 27 (28\%) & 20 (17\%) & 17 (24\%) & 25 (27\%) & 37 (39\%) \\
## Type of challenges & & & & & & & & & & \\
## No impact & 10 (9.9\%) & 20 (21\%) & 27 (29\%) & 4 (3.9\%) & 37 (45\%) & 20 (21\%) & 15 (13\%) & 25 (35\%) & 11 (12\%) & 14 (15\%) \\
## Other & 11 (11\%) & 37 (39\%) & 10 (11\%) & 21 (20\%) & 9 (11\%) & 37 (39\%) & 19 (16\%) & 19 (26\%) & 39 (41\%) & 41 (44\%) \\
## School & 80 (79\%) & 39 (41\%) & 57 (61\%) & 78 (76\%) & 37 (45\%) & 38 (40\%) & 86 (72\%) & 28 (39\%) & 44 (47\%) & 39 (41\%) \\
## Type of coping strategies & & & & & & & & & & \\
## Combined & 0 (0\%) & 31 (32\%) & 0 (0\%) & 0 (0\%) & 2 (2.4\%) & 1 (1.1\%) & 8 (6.7\%) & 15 (21\%) & 29 (31\%) & 25 (27\%) \\
## No action & 16 (16\%) & 65 (68\%) & 28 (30\%) & 33 (32\%) & 36 (43\%) & 63 (66\%) & 95 (79\%) & 49 (68\%) & 50 (53\%) & 29 (31\%) \\
## Positive & 85 (84\%) & 0 (0\%) & 66 (70\%) & 70 (68\%) & 45 (54\%) & 31 (33\%) & 17 (14\%) & 8 (11\%) & 15 (16\%) & 40 (43\%) \\
## person\_ate & & & & & & & & & & \\
## -0.0689513014845681 & 101 (100\%) & 96 (100\%) & 94 (100\%) & 103 (100\%) & 83 (100\%) & 95 (100\%) & 120 (100\%) & 72 (100\%) & 94 (100\%) & 94 (100\%) \\
## person\_hte & -0.10 (0.01) & -0.03 (0.01) & -0.09 (0.00) & -0.08 (0.00) & -0.08 (0.00) & -0.07 (0.00) & -0.07 (0.00) & -0.06 (0.00) & -0.06 (0.00) & -0.04 (0.00) \\
## lp\_person & 0.84 (0.03) & 1.17 (0.03) & 0.89 (0.01) & 0.93 (0.01) & 0.94 (0.01) & 0.96 (0.01) & 1.00 (0.01) & 1.01 (0.01) & 1.05 (0.01) & 1.10 (0.02) \\
## \hline
## \end{tabular}
## \end{table}
6 Sensitivity Analysis
6.1 inverse probability of missingness weighting (IPMW)
Check the missingness, and regress the missing status on variables with complete info.
## .
## 0 1
## 1441 60
## tibble [1,501 × 10] (S3: tbl_df/tbl/data.frame)
## $ condition : chr [1:1501] "Placebo Control" "Project Personality" "Placebo Control" "Placebo Control" ...
## $ b_cdi_mean : num [1:1501] 1.417 0.917 1.167 1.333 0.917 ...
## $ f1_cdi_mean : num [1:1501] 1 0.583 0.583 1.417 0.5 ...
## $ recode_race : Factor w/ 5 levels "Asian Including Asian Desi",..: 5 5 5 5 5 5 1 1 5 4 ...
## $ b_dem_sex : Factor w/ 2 levels "Female","Male": 1 1 1 1 1 1 1 1 1 1 ...
## $ recode_orient: Factor w/ 3 levels "Heterosexual",..: 2 3 2 2 2 2 1 2 1 2 ...
## $ genderid3 : Factor w/ 3 levels "1","2","3": 1 1 2 2 2 3 2 2 2 2 ...
## $ family_cat : Factor w/ 3 levels "No impact","Other",..: 2 2 1 3 3 3 3 3 3 2 ...
## $ cope3 : Factor w/ 3 levels "1","2","3": 3 3 3 2 1 2 2 2 2 2 ...
## $ missing : num [1:1501] 0 0 0 0 0 0 0 0 0 0 ...
A summary of the IPCW model:
##
## Call:
## glm(formula = missing ~ genderid3 + b_cdi_mean + family_cat +
## cope3, family = "binomial", data = model_dat)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -3.0919 0.8430 -3.668 0.000245 ***
## genderid32 -1.2368 0.2870 -4.309 1.64e-05 ***
## genderid33 -1.2284 0.4630 -2.653 0.007978 **
## b_cdi_mean 0.2561 0.4193 0.611 0.541408
## family_catOther -0.1730 0.3659 -0.473 0.636328
## family_catSchool -0.5263 0.3270 -1.610 0.107490
## cope32 0.9205 0.5452 1.688 0.091346 .
## cope33 0.7123 0.5729 1.243 0.213729
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 503.91 on 1500 degrees of freedom
## Residual deviance: 476.46 on 1493 degrees of freedom
## AIC: 492.46
##
## Number of Fisher Scoring iterations: 6
Summary of IPCW weights:
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.9687 0.9815 0.9878 1.0000 1.0002 1.1427
6.2 Baseline prediction models
baseline model for Project ABC:| f 1 cdi mean | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 1.06 | 0.92 – 1.21 | <0.001 |
|
recode race [Black/African-American] |
-0.04 | -0.16 – 0.08 | 0.527 |
|
recode race [Hispanic/Latinx] |
-0.08 | -0.19 – 0.02 | 0.128 |
| recode race [Mixed] | -0.03 | -0.13 – 0.08 | 0.617 |
| recode race [White] | 0.03 | -0.05 – 0.12 | 0.462 |
| b dem sex [Male] | -0.15 | -0.25 – -0.04 | 0.006 |
| recode orient [LGBTQ] | 0.09 | 0.02 – 0.16 | 0.010 |
| recode orient [Other] | 0.12 | 0.03 – 0.22 | 0.008 |
| genderid3 [2] | -0.13 | -0.20 – -0.06 | <0.001 |
| genderid3 [3] | -0.02 | -0.12 – 0.08 | 0.687 |
| family cat [Other] | 0.07 | -0.01 – 0.15 | 0.070 |
| family cat [School] | 0.03 | -0.03 – 0.10 | 0.313 |
| cope3 [2] | -0.06 | -0.14 – 0.02 | 0.137 |
| cope3 [3] | -0.16 | -0.24 – -0.08 | <0.001 |
| Observations | 977 | ||
| R2 | 0.079 | ||
| f 1 cdi mean | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 1.16 | 1.00 – 1.31 | <0.001 |
|
recode race [Black/African-American] |
-0.06 | -0.18 – 0.07 | 0.387 |
|
recode race [Hispanic/Latinx] |
-0.05 | -0.16 – 0.06 | 0.382 |
| recode race [Mixed] | -0.03 | -0.14 – 0.08 | 0.552 |
| recode race [White] | -0.00 | -0.09 – 0.09 | 0.937 |
| b dem sex [Male] | -0.11 | -0.22 – 0.01 | 0.065 |
| recode orient [LGBTQ] | 0.07 | 0.00 – 0.14 | 0.046 |
| recode orient [Other] | 0.06 | -0.03 – 0.16 | 0.182 |
| genderid3 [2] | -0.15 | -0.22 – -0.08 | <0.001 |
| genderid3 [3] | -0.10 | -0.21 – -0.00 | 0.049 |
| family cat [Other] | 0.02 | -0.06 – 0.10 | 0.625 |
| family cat [School] | -0.02 | -0.09 – 0.05 | 0.664 |
| cope3 [2] | -0.06 | -0.15 – 0.03 | 0.193 |
| cope3 [3] | -0.13 | -0.22 – -0.04 | 0.005 |
| Observations | 952 | ||
| R2 | 0.052 | ||
6.3 Main effect model
Main effect model for Project ABC:| f 1 cdi mean | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 0.27 | 0.19 – 0.36 | <0.001 |
| b cdi mean | 0.64 | 0.58 – 0.71 | <0.001 |
| condition [Project ABC] | -0.08 | -0.12 – -0.04 | <0.001 |
| Observations | 977 | ||
| R2 | 0.292 | ||
| f 1 cdi mean | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 0.31 | 0.23 – 0.39 | <0.001 |
| b cdi mean | 0.61 | 0.55 – 0.68 | <0.001 |
|
condition [Project Personality] |
-0.07 | -0.11 – -0.02 | 0.003 |
| Observations | 952 | ||
| R2 | 0.266 | ||
6.4 The HTE models
HTE model for Project ABC:| f 1 cdi mean | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 0.21 | -0.05 – 0.47 | 0.120 |
| b cdi mean | 0.60 | 0.53 – 0.67 | <0.001 |
| condition [Project ABC] | -0.46 | -0.83 – -0.10 | 0.013 |
| abc ipcw lp | 0.12 | -0.15 – 0.39 | 0.393 |
|
condition [Project ABC] × abc ipcw lp |
0.39 | 0.02 – 0.77 | 0.039 |
| Observations | 977 | ||
| R2 | 0.302 | ||
| f 1 cdi mean | |||
|---|---|---|---|
| Predictors | Estimates | CI | p |
| (Intercept) | 0.19 | -0.14 – 0.52 | 0.258 |
| b cdi mean | 0.59 | 0.52 – 0.66 | <0.001 |
|
condition [Project Personality] |
-0.28 | -0.76 – 0.19 | 0.242 |
| person ipcw lp | 0.15 | -0.19 – 0.49 | 0.383 |
|
condition [Project Personality] × person ipcw lp |
0.22 | -0.26 – 0.70 | 0.373 |
| Observations | 952 | ||
| R2 | 0.270 | ||
| Comparsion | ATE | SE | |
|---|---|---|---|
| conditionProject ABC | Project ABC vs Control | -0.0795347 | 0.0223413 |
| conditionProject Personality | Project personality vs Control | -0.0689513 | 0.0233696 |
6.5 Risk stratification
6.6 Boot CI
For Project ABC:
For Project personality:
6.7 Risk stratification
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.6620 0.8980 0.9743 0.9782 1.0605 1.2931
Distribution of baseline depression severity score
7 Appendix
The original data summary statistics:
| Demographics | Treatment Received | ||
|---|---|---|---|
| Placebo Control N = 5051 |
Project ABC N = 5141 |
Project Personality N = 4821 |
|
| recode_race | |||
| Asian Including Asian Desi | 50 (9.9%) | 59 (11%) | 50 (10%) |
| Black/African-American | 33 (6.5%) | 41 (8.0%) | 36 (7.5%) |
| Hispanic/Latinx | 57 (11%) | 62 (12%) | 55 (11%) |
| Mixed | 76 (15%) | 70 (14%) | 63 (13%) |
| Prefer not to answer | 12 (2.4%) | 9 (1.8%) | 12 (2.5%) |
| White | 277 (55%) | 273 (53%) | 266 (55%) |
| Agender | 11 (2.2%) | 7 (1.4%) | 11 (2.3%) |
| Not sure/Questioning | 39 (7.7%) | 31 (6.0%) | 32 (6.6%) |
| Unspecified Gender | 15 (3.0%) | 13 (2.5%) | 12 (2.5%) |
| Androgynous | 27 (5.3%) | 28 (5.4%) | 31 (6.4%) |
| Non-binary | 68 (13%) | 71 (14%) | 68 (14%) |
| Two-spirited | 0 (0%) | 4 (0.8%) | 4 (0.8%) |
| Transgender - Female to Male | 36 (7.1%) | 35 (6.8%) | 38 (7.9%) |
| Trans Female/Trans Feminine | 6 (1.2%) | 8 (1.6%) | 3 (0.6%) |
| Trans Male/Trans Masculine | 40 (7.9%) | 38 (7.4%) | 38 (7.9%) |
| Gender Expansive | 3 (0.6%) | 6 (1.2%) | 4 (0.8%) |
| Third Gender | 1 (0.2%) | 1 (0.2%) | 2 (0.4%) |
| Genderqueer | 33 (6.5%) | 22 (4.3%) | 23 (4.8%) |
| Transgender - Male to Female | 4 (0.8%) | 3 (0.6%) | 3 (0.6%) |
| Man/Boy | 75 (15%) | 69 (13%) | 77 (16%) |
| Transgender | 45 (8.9%) | 33 (6.4%) | 38 (7.9%) |
| Woman/Girl | 329 (65%) | 337 (66%) | 323 (67%) |
| b_screener_age | |||
| 13 | 28 (5.5%) | 36 (7.0%) | 29 (6.0%) |
| 14 | 77 (15%) | 85 (17%) | 65 (13%) |
| 15 | 158 (31%) | 164 (32%) | 170 (35%) |
| 16 | 242 (48%) | 229 (45%) | 218 (45%) |
| Biological Sex | |||
| Female | 445 (88%) | 449 (87%) | 429 (89%) |
| Male | 55 (11%) | 54 (11%) | 47 (9.8%) |
| Other | 2 (0.4%) | 8 (1.6%) | 2 (0.4%) |
| Prefer not to say | 3 (0.6%) | 3 (0.6%) | 4 (0.8%) |
| Sexual Orientation | |||
| Asexual | 24 (4.8%) | 29 (5.6%) | 28 (5.8%) |
| Bisexual | 143 (28%) | 144 (28%) | 123 (26%) |
| Gay/Lesbian/Homosexual | 55 (11%) | 52 (10%) | 53 (11%) |
| Heterosexual/Straight | 110 (22%) | 101 (20%) | 107 (22%) |
| I do not use a label | 33 (6.5%) | 29 (5.6%) | 24 (5.0%) |
| I do not want to respond | 0 (0%) | 5 (1.0%) | 1 (0.2%) |
| Other/Not listed (please specify) | 18 (3.6%) | 12 (2.3%) | 20 (4.1%) |
| Pansexual | 49 (9.7%) | 58 (11%) | 41 (8.5%) |
| Queer | 34 (6.7%) | 28 (5.4%) | 28 (5.8%) |
| Unsure/Questioning | 39 (7.7%) | 56 (11%) | 57 (12%) |
| b_covid_family_family_did_not_enough_enough_money_for_food | 51 (10%) | 43 (8.4%) | 55 (11%) |
| b_covid_family_family_did_not_have_a_regular_place_to_sleep_or_stay | 7 (1.4%) | 5 (1.0%) | 6 (1.2%) |
| b_covid_family_i_could_not_attend_school_in_person | 358 (71%) | 367 (71%) | 330 (68%) |
| b_covid_family_i_could_not_attend_school_at_all | 45 (8.9%) | 32 (6.2%) | 36 (7.5%) |
| b_covid_family_other | 56 (11%) | 39 (7.6%) | 56 (12%) |
| b_covid_family_family_did_not_have_enough_money_for_gas_transportation | 36 (7.1%) | 20 (3.9%) | 32 (6.6%) |
| b_covid_family_family_did_not_have_enough_money_to_pay_rent | 43 (8.5%) | 36 (7.0%) | 41 (8.5%) |
| b_covid_family_the_covid_19_pandemic_has_not_affected_me_or_my_family_in_these_ways_in_the_past_2_weeks | 100 (20%) | 107 (21%) | 91 (19%) |
| b_covid_cope_1_connecting_with_others | 233 (46%) | 227 (44%) | 205 (43%) |
| b_covid_cope_1_including_talking_with_people_you_trust_about_your_concerns_and_how_you_are_feeling | 233 (46%) | 227 (44%) | 205 (43%) |
| b_covid_cope_1_contacting_a_healthcare_provider | 58 (11%) | 51 (9.9%) | 49 (10%) |
| b_covid_cope_1_drinking_alcohol | 37 (7.3%) | 42 (8.2%) | 33 (6.8%) |
| b_covid_cope_1_smoking_more_cigarettes_or_vaping_more | 35 (6.9%) | 37 (7.2%) | 33 (6.8%) |
| recode_language | |||
| English | 493 (98%) | 499 (97%) | 468 (97%) |
| Other | 12 (2.4%) | 15 (2.9%) | 14 (2.9%) |
| b_cdi_mean | 1.16 (0.35) | 1.16 (0.34) | 1.17 (0.35) |
| recode_orient | |||
| Heterosexual | 110 (22%) | 101 (20%) | 107 (22%) |
| I do not want to respond | 0 (0%) | 5 (1.0%) | 1 (0.2%) |
| LGBTQ | 320 (63%) | 338 (66%) | 302 (63%) |
| Other | 75 (15%) | 70 (14%) | 72 (15%) |
| 1 n (%); Mean (SD) | |||